Skip to content

Add structured request-content parsing to Egress Gate - #35

Merged
johnnygreco merged 11 commits into
mainfrom
johnny/egress-gate-request-content-parsers
Aug 8, 2026
Merged

Add structured request-content parsing to Egress Gate#35
johnnygreco merged 11 commits into
mainfrom
johnny/egress-gate-request-content-parsers

Conversation

@johnnygreco

@johnnygreco johnnygreco commented Aug 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • add reusable request-content parsers for complete UTF-8 bodies, selected JSON fields, and normalized message blocks
  • let the regex gate detect, deny, and source-preservingly replace selected structured content while retaining complete-body behavior
  • enforce the 4 MiB input/output boundary directly at the public parser API
  • preserve distinct message-block classifications through filtering while deduplicating identical (node, kind) classifications
  • document the parser architecture, configuration, limits, overlap semantics, and replacement behavior

Why

Egress Gate previously treated a request body as one complete UTF-8 string. Agent harness requests need bounded, configurable extraction of model-visible JSON fields and message blocks without scanning unrelated metadata or losing exact source formatting during replacement.

QA also found two boundary issues in the first implementation: the reusable parser API relied on downstream request models for some body-size enforcement, and a broad text selector could shadow an explicit tool classification before block-kind filters ran. The final implementation owns its advertised bounds and filters distinct classifications before exposing each backing string once.

Impact

Policies can opt into json-fields or message-blocks scans. Existing body, path, query, and header behavior remains available. Structured replacement rewrites only selected JSON string tokens and preserves all unrelated request bytes. Explicit tool-input and tool-output filters cannot be bypassed by an overlapping generic text selector.

QA and validation

  • make check — 324 tests plus formatting, linting, typing, installed-package import, and dependency audit
  • Python 3.11 focused parser/regex suite — 83 tests
  • python3 tests/test_render_dev_notes.py — 11 tests
  • scripts/build-docs.sh — strict build, 14 agent-readable pages, all documentation tests pass
  • isolated adversarial parser QA — 2,000 generated valid documents, 25,000 strict-JSON mutation comparisons, 4,000 concurrent parses, exact resource boundaries, replacement ordering, and classification-limit probes
  • isolated end-to-end QA — source and installed-wheel CLI, generated schema, policy validation/evaluation, direct processor, live gRPC error/recovery and concurrency, all scan/action variants, and pipeline chaining
  • isolated documentation/adoption QA — documented snippets and policies, installed workflows, strict site build, served HTML/Markdown routes, schema/runtime cross-checks, and limit/error wording
  • final multi-agent rerun — parser, end-to-end, and documentation lenses all clean after fixes

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Documentation preview

The preview has been removed.

@johnnygreco
johnnygreco marked this pull request as ready for review August 8, 2026 03:16
@johnnygreco
johnnygreco merged commit 3f9a5cd into main Aug 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant